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INTRODUCTION 

This   study,  which   is   part   of  a  larger  study  of  project  management, 
(Marquis   and  Straight,    [3];   Rubin   and  Marqtiis    [T];   Rubin    [6])   was   car- 
ried out   to  investigate  the  effects   of  Program  Evaluation   and  Review 
Technique    (PERT)   on   large  R&D  organizations.      Since  PERT  is  primarily 
a  scheduling  system,  prior  research  has    focused  on   its  effectiveness 
in   solving  scheduling  problems;    (see,   for  example,    [h])   in   addition, 
there  has  been  some   research   concerning  management  use   and  acceptance 
of  PERT.      Hayya,    (Hayya   [l])    for  instance,  has   done   an  exhaustive   study 
on  the   implementation   of  PERT  by  the   Department   of  Defense.      There  was, 
apparently,   much  bitterness   generated  as   a  result  of  this   implementation, 
but   as  Hayya  suggests   "Much  of  the   friction  between  the  government   and 
the   aerospace   indxistry  attributed  to  PERT  is   really   due   to  the   older, 
more  subtle   and  extremely  more   complex  marriage  between  the   defense 
establishment  and  the  aerospace  industry".      (Hayya,   1966,  pages  2-3) 
Hayya  derives   a  decision  model,  based  upon  variables   such   as   type   and 
size   of  contract,  by  means   of  which  a  procurement  manager  may   decide   to 
require  the  use   of  PERT.      Although  this   type   of  research  is   extremely 
valuable,   it   does   leave   some  questions   unanswered. 

For  example,  Hilton,    (Hilton,    [2])   in   a  study   of  seven   companies 
not  primarily  engaged  in  Research   and  Development    found  that   as   far  as 
the   individual   companies  were   concerned  the   reason   for  using  PERT  is  not 
always   solely  to  improve  scheduling:      "The  most   frequently   cited  advantage 
was   not   savings   in   cost   or  time  but   rather  improved  communications   and 


cooperation  between  groups   involved  in   a  project".      (Hilton,   1966, 
pages   28)      Hilton   also  notes   at  this   point   that   the  improved  commun- 
ications  are  often   due  to  the   fact   that   in  order  to  set   up  the  PERT 
chart  the  man   in   charge   of  the  project  must  become   familiar  with   all 
stages   of  the   job,   and  that   in  the  planning  meetings  before  the  Job 
begins  the   individuals   in   charge   of  the  various   subactivities  become 
acquainted  with  each  other's  problems.      Convincing  people   in   an   organ- 
ization that  PERT  can  be  helpful  is   also  a  problem:      "Introducing  the 
method  often  required  a  soft   sell,    and  most   firms   agree  that   it  takes 
up  to   five  years   to  obtain   complete   acceptance."      (Hilton,   I966 ,  pages    32) 

Poust,   in  a  study   of  large   scale  government  sponsored  R&D  projects 
(Poiist   and  Rubin,    [5])   has  pointed  out   an   interesting  anomaly  in  the   roles 
of  the  project  manager  and  the  lab  manager.   They  were   asked  to  list   the 
most   important  problems   facing  a  person   in  their  position.      They  were  also 
asked  to  estimate  how  they  spent  their  time   on  the  project   under  investi- 
gation  across   a  series   of  comparable  problem  areas.    Neither  the  project 
nor  the  lab  managers   spend  their  time   on  those  problems  which  they   con- 
sider to  be   generally  most   important.      For  example,   althovigh   across  the 
sample   of  projects   studied  they  both   spend  most  of  their  time   fighting 
"fires",    (technical,   cost,    and  schedule),   the  project  manager  considers 
finding  methods   of  controlling  these  problems  by   far  the  most   important 
aspect  of  his   job  while   the  lab  manager  thinks   that   finding  and  keeping 
competent  people   and  providing  motivation   for  those   currently  working  on 
projects   are  the  most   important  aspect  of  his   job. 
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In  the   light   of  these  kinds   of  evidence   it   is   reasonable  to  ask 
several  kinds   of  questions   about  what  the  introduction  of  PERT  vill  do 
to  an   R&D  organization.      Will  the   use  of  PERT  create   conditions   favor- 
able to  better  commxini  cat  ions?     Will  there  be   resistance  to  its  use? 
Will  the  use  of  PERT  cause   changes   in  authority   structures   that  will 
tend  to  bring  the  actual  problem  structures  which  the   lab  manager  and 
project  manager  face  more   in   line  with  the  ones  which  they   consider 
to  be   generally  most   important?      In   addition  to  these   organizational 
issues,  questions   regarding  the  impact  of  utilizing  PERT  on  project 
performance  will  also  be   examined. 

METHOD 
Sample  Characteristics 

A  project  was   selected  in   a  company   or  a  government   agency   on  two 
criteria:      (l)   random  or  most   recently   completed,   and   (2)   more  than  $1 
million   in  size.      The  projects   were   contracted  by  twelve  government 
agencies.      The   firms   studied  are   all  in   the   aerospace   and  electronics 
industries ,    and  80  percent   of  them  are   among  the   100   largest  performers 
of  government- funded  R&D.      They   are  located  in   all  parts   of  the   country. 

The  projects   ranged  in   size   from  $1  million  to  $60  million  with   a 
median  of  $h  million.      The   average  project    duration  was    3.*+  years   and 
none   lasted  more  than  6  years.      Almost   all   of  the  projects   required  ad- 
vances  in  the   "state   of  the   art"   in   a  technological   field  such   as   advanced 
radar  systems,  microminiaturization   of  electronics  modules,   electronic 
data  processing  interfaces  with  telemetry   systems,  etc.      The  projects 
studied  are  more   developmental  than   fundamental  research. 
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Information  on  each  project  was  obtained   from   five   sources  :      the 
laboratory  manager   (LM),   the  project  manager   (PM) ,  the  government 
technical  monitor      (CM),   the  government   contract   administrator   (GCA)  , 
and  the   company   contract   administrator   (CA).      Although  a  total   of  U8 
projects  were   studied,   inevitable   difficulties   in   securing  some   of  the 
desired  information   resulted  in   incomplete   records   in  several   cases. 
In   73%  of  the   cases  the   records   are   complete    (LM,   CA,   PM,  GTM,   and  GCA). 
Incomplete   information   results  primarily   from  an   inability  to   interview 
either  the   LM,   GTM,   or  the  GCA.      Pert  was   used  on   2k  of  these  projects 
and  other  methods   such   as  bar  charts,   gantt    charts        and  milestone 
identification  were   used  on  l8:      scheduling   information  was  not   available 
on  the   remaining  6. 

Cost   and  Schedule   Performance 

Information  on   cost   and  schedule   changes  was  gathered   from  the  project 
manager  and  contract   administrator  in  the   company   and   from  the  government 
contract   administrator.      Careful   cross-checking  on  the  reasons   for  contract 
changes   made   it   possible  to  eliminate  those   caused  by   a  customer's   action, 
such   as   a  change  in   scope   or  specifications,   or  the  vinavai lability  of  gov- 
ernment test   equipment   or  facilities.      Thus   only  those   slippages  which 
were   directly   attributable   to  the   actions   of  the   contractor  firm  are   in- 
cluded.     While  the   data  do  not   at  present   include  the  amoxmt   of  cost   and 
schediile   variance,   only  those  greater  than   10  percent   are   included.      On 
this  basis  there  were   28  overruns   and  l6  on   target   or  underruns.      The   rank 
correlation  between   cost   and  schedule  performance  was   0.82,   so  the  two   are 
combined  to   form  a  single  performance  measure. 


Technical   Performance 

It   is   presently  impossible  to   compare  the  technical  performance  of 
different  projects  by   any   objective  measures.      In  one   instance,   speed 
may  be  the  primary   technical  objective   of  a  system   (missile,    airplane, 
etc.),   while   in   another  case   range   is  most    critical.      In   an  electronics 
system,    reliability   or  maintainability  may  be   the   chief  goal.      Consequently 
the  measure   of  success f\il  technical  performance   used  in  this   study  are 
expert   judgments  by  the   most   fully  informed  individuals. 

Success   ratings  were   obtained  independently   from  the   project   manager, 
the  laboratory   manager,   the   government   contract   administrator  and  the 
technical  monitor.      The  ratings  were   on   a  scale   from  one   to  nine,   with 
nine   representing   an   outstanding  success    and  one  signifying  a  failure 
(in  some   relative  sense,   since  no   project   in  this   study   achieved  the   ab- 
solute  failure   of  being  cancelled  before   completion). 

The  measure   of  performance   used  in  this   study   is  the  average  of  the 
ratings  provided  by  the  project   manager  and  the  government  technical  mon- 
itor,  plus    a  constant   term  to  account    for  the   fact  that  project  managers 
were   consistently  more   optimistic  than  technical  monitors.    All     except 
one   of  these   scores    fell   in  the  range   of  five  to  nine  within  which  there 
was    a  symmetrical   distribution  with  the  median   at  6.U.      For  the  sample   of 
projects   studied,   technical  performance   is   unrelated  to   cost   and  schedule 
performance. 
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Aiithority  Structure 

In   order  to  get  some   idea  of  the  authority  structure   of  the   organi- 
zation, both  the   LM  and  the   PM  were   asked  a  number  of  questions   concerning 
decision  making  power  in   various   areas.      (See  table   l) 

On   any  given  project  there   are  three   levels  within  the   laboratory  at 
which   decisions  may  be  made:      they  may  be  made  by  the   project   manager's 
subordinates,  by  the  project  manager,   or  by  laboratory  management.      The 
first   of  these  two   are   considered  within  the  project   area  and  the  latter 
oxitside   the  project   area. 

Communication 

The  kind  of  communication  which  we   are  trying  to  measure   is   essentially 
informal   rather  than   formal.      That   is  ,  we   are  trying  to  measure  the  effec- 
tiveness  of  existing  communication   links,   in  particular  the   FM-LM  link, 
rather  than   looking  at  the   communication   structure.      Any  such  process   mvist, 
at   the  present  time,   be   inferential.      The  method  which  we  have   used  is  to 
call  the  LM  rating  of  PM  technical  performance   and  informal  communication 
measure.      More  specifically,  we  expect  that   on  those  projects   using  PERT, 
a  LM  will  think  that  his   PM  is   doing  a  superior  job.      This    feeling  on  the 
part   of  the  LM  will  be   independent   of  how  well  the  PM  is   actually   doing. 

This   expectation   rests   upon  the   assumption,   substantiated  by  Hilton's 
work  mentioned  earlier,   that  the   LM  and  PM  will  better  \mderstand  each 
■'".her's   pro'hlems   as   a  result   of  the  planning  meetings   associated  with  the 
utilization   of  PERT. 


Table   1 

Decision   Level  Questions   Asked  of 
Project  Manager  and  Laboratory  Manager 


■      — 1 

Who 

in  the  Organization  Made  the  Following 
Decisions    for  this   Project? 

PM'S 
SUBORDINATES 

PM 

HIGHER 

1. 

Authorization   of  total  overtime  budget 

?. 

Contract   change   in   schedule  or  cost 

3. 

Authorization  to  exceed  company   funding 

k. 

Exceed  approved  personnel  rushing   for 
crashing  project 

5. 

Initiation  of  work  in  support   ureas 

6. 

Assign  priority  of  work   ^i  support   areas 

7. 

Change  schedules    for  project   subactivities 

8. 

Make  versus  b\:iy   (whether  to  subcontract) 

9. 

Select  subcontractors 

10. 

Select   sources   of  supply    (off  shelf  items) 

11, 

Hire   additional  people 

12. 

Contract   change   in  technical   scope 
(or  content) 

13. 

Bring   subcontracted  work   in-house 

11*. 

Authorize   subcontractors  to  exceed   cost 
or  schedule 

15. 

Authorize   subcontractor  to   reduce   technical 
content 

16. 

Omit   tests 

IT. 

Relax  performance   requirements 

18. 

Create   additional   concurrent   schedules 
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RESULTS 

The  analysis  procedure  was  to  use  the  l8  projects  which  did  not 
use  PERT  as  a  "control"  group  and  compare  them  with  the  2^  which  did. 
In  order  to  control  for  any  contaminating  influences  in  the  analysis, 
the  data  were  first  checked  to  see  if  the  results  were  due  to  any  ob- 
viously exogenous  sources.  The  first  two  sources  which  were  checked 
were  dollar  size  and  degree  of  haste  i.e.,  whether  or  not  it  was  a 
crash  program.      The   reason   for  checking  these  two  particular  areas  is 

Table   II 
Utilization   of  PERT  vs.    Dollar  Size   of  Project 


1 

SIZE/10^ 


NUMBER 

NUIffiER  OF 

OF  PROJECTS 

PERT 

PROJECTS 

%  PE 

5 

3 

605S 

6 

2 

33^ 

8 

1^ 

505? 

6 

U 

(>^% 

6 

3 

30% 

1 

2 

3-1+    (INCL.) 
6-10    (INCL.) 
>10 


31 


15 


Fisher  Exact    (SPLIT  ON  MEDIAN)     <.1+1 


Thro\:ighout   the   analysis   a  significance   level  of   .05    (one  tail)  was   \ised. 


Table  III 
PERT  vs .    Crash  Programs 


CRASH 


NON   CRASH 


PERT 


NON   PERT 


3i*^ 

59% 

66% 

hi% 

N=26  N=1T 

FISHER  EXACT    <.15 


that   either  of  these    factors  may  well   affect   the   internal  priority  which 
the  organization   gives  to  the  project.      Rubin    (Rubin,    [6])  has    foimd 
that   technical  performance   is    critically  sensitive  to  internal  priority. 
However,   it  is   readily  obvious    (see   tables   II   and  III)   that  PERT  is  not 
particularly  associated  with  either  high  or  low  project   dollar  value   or 
crash   or  non-crash  priority.      The  other  variable  which  we   controlled  for 
was   the  date  the   project  started.      To  understand  the  necessity   for  checking 
this   variable  we  must   view  the  historical  perspective  both  of  the   data 
and  of  the  use   of  PERT.      The  time  span  of  the   data,  1957  to  1963,    coincides 
roughly  with  the  period  of  time   during  which  PERT  was   introduced.      If  it 
happened  that   all  of  our  PERT  projects  had  begun,   say,   in  the   last    3  years, 
then  our  results   could  be   open  to  the  interpretation  that   the  effects   are 
due  to  a  time   displacement   rather  than  to  PERT  itself.      However  this   is   not 
the   case.      (see  Table   IV) 
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Table  IV 
Utilization  of  PERT  vs.    Year  Project  Started 


YEAR 


NUMBER  OF 
PROJECTS  STARTED 


% 
PERT  PROJECTS 


1957 
1958 
i960 
1961 
1962 
1963 


1 
2 
5 
12 
7 
9 


0% 
50^ 
UO^ 

58^ 

57?^ 
67^ 


Pert  vs.    Perfo I'^^-an^ 

There   is  no  significant   difference   in  technical  performance  be- 
tween projects  which   used  PERT  and  those  which   didn't.      (See   Table   V) 

Table   V 
Relation  Between   PERT  and  Technical  Performance 


%   GREATER  THAN  OR  EQUAL  TO  f^EDIAI-I 

PERT 

NON  PERT 

TECHNICAL 

56? 

59:^ 

PERFORMANCE 

%   LESS  THAN  MEDIAN 

\^t 

^n 

N=2U 


N=l8 


MANN  WHITNEY  U  NOT  SIGNIFICANT 
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There   is  however,    a  significant   difference   in   cost/schedule  per- 
formance between  those  projects  which  use   PERT   and  those  which   do  not. 
(Table   VI)   Pert  projects   do  better  in   cost   and  schedxile  performance  than 
do  non-pert  projects.      This  will  not   come   as    a  great   shock  to  those  who 
have  previously  believed  in   PERT   as   a  scheduling  tool. 

Table  VI 
Relation  Between  PERT   and  Schedule  Performance 


PERT 

NON  PERT 

%   OVERRUNS 

h2% 

67% 

%   NON  OVERRUTJS 

5Q% 

33^ 

N=20         K=l6 
FISHER  EXACT  PROBABILITY  <.05 


Pert   vs.    Authority  Structure 

As  might  be  expected,   it   is   more   difficult  to  determine   differences 
in  attthority  structure  than  it   is  to  find  differences  in  technical  and 
cost /schedule  performance.      The   approach  we   used  here  was  to  compare  the 
lab  managers'    responses   to  the  decision   level  question    (see  Table   l)    on 
PERT  projects   to  the  responses   on  non-PERT  project.      There  were   five 
significant   differences.      (see  Table   VIl).      For  example,   the   LM  is   much 
more   likely  to  think  he  has  the   authority  to   change   schedules    for  project 
subactivities   if  he   is  working  on   a  PERT  project   than  he   is   if  he  is 
working  on   a  non-PERT  project. 
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Table  VII 
Lab  Manager  Perceived  Authority  Structure  on  PERT  Projects 


DECISION  AREA 


AUTHORITY  LEVEL  IS: 


IN  THE 
PROJECT  AREA 


OUTSIDE  THE 
PROJECT  AREA 


ESTABLISH  PRIORITY 

AUTHORIZE  OVERTIME  BUDGET 

MAKE  THE  DECISION  TO  SUBCONTRACT 

INITIATE  WORK  IN  A  SUPPORT  AREA 

CHANGE  SCHEDULES  FOR  PROJECT 
SUBACTIVITIES 


« 
« 


As  was   mentioned  previously,    "in  the  project    area"  means  either  with 
the  project  manager  or  his   subordinates;    "outside   the  project   area"  means 
with  the  lab   manager.      As    a  check  on  the   resiilts  ,  the   answers  which  the   PM 
gave   to  the   same   question  were   also   checked.      In   all   cases   except   one,   there 
was   agreement:      the  exception  was    "establish  priority."      In  this   instance, 
each  thought   that  the  other  had  authority  in  this   area. 

Communication 

The   informal   communication  measure  which  we  have   used  is   LM  rating 
of  PM  technical  performance.      As   anticipated  the  lab  managers   on   PERT 
projects   give   considerably  better  performance   ratings   to  their  project 
managers  than   do  those   on  non-PERT  projects.      (See  Table  VIII)      Validation 
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Table   VIII 
Lab  Manager  Rating  of  Project   Manager  Performance   vs.    PERT 


PERT 

NO'.  PERT 

i   RECEIVING  EXCELLENT  RATING 

61^ 

29^ 

%   RECEIVING  LESS  THAN  EXCELLENT 

'i9l' 

1\% 

RATING 


N^23  N^17 

FISHER  EXACT     < .05 


of  the   assumption  is    'one  by   comparing  the   LM  rating  of  PM  technical 

performance  vith   actual  pro,1ect   performance.      The   results   are   that   in 

fact  the  LM  rating  is   not    related  to   actual  project   perfornance.      (Table   IX) 

As  was   mentioned  previously,   the   process   of  informal   communication 
measurement   must  be   inferential.      The   inference  which  we   draw   from  this 
somewhat   surprising  result   n  r>   that  ^■^'^■^or\  the  T.M  answered  this   question  he 
was   not   thinking   about   hov;  well  the  PM  actually  performed  technically, 
hut  how  i'ell  he    (the  LM)    covild  relate   to  the   research  process   through  the 
?M,    and  this   is    a  measure   of  communication.      The   lab  manager's   rating  of 
the  project  manager's    administrative   ability  was    unrelated  to   actual   cost 
and  schediile  performance  which   further  strengthens   this    argument. 
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Table    IX 


Tech  Performance   vs.    LM  Rating  of  PM  Tech  Performance 


%   GREATER 
THAN    (OR  EQUAL  TO)    MEDIAN 

LM  RATING 
DF  EXCELLENT 

LM  RATING   OF 
LESS  THAN  EXCELLENT 

TECHNICAL 

k2% 

525g 

PERFORMANCE 

%  LESS   THAN  MEDIAN 

5S% 

k&% 

N=19 


N=21 


MANN  WHITNEY  U  NOT  SIGNIFICANT 


CONCLUSIONS   AND  DISCUSSION 


There   are  two   definite   conclusions  which   can  be   drawn  from  the   results. 
First,   PERT  does   lead  to  improvement   of  schedule  performance   and  secondly, 
it   does   so  without   any  noticeable   effect   on  technical  performance.      The 
other  results   shed  some   light   on  how  projects   that   use  PERT   differ  from 
those  that   do  not,   and  provide   some  basis    for  speculating  on  how  the  im- 
provement  is   achieved. 

The   improvement   in   communication  with  the  use   of  PERT  is   indicative   of 
an  increase   in   lab  manager  visibility  to  the  project   manager.      And  visa  versa 
it   is   interesting  to  note  that  the   areas   in  which  the  LM  is   more   visible, 
i.e.  ,   in   initiating  work   and  changing  schedules,  both   lie   in  the   schedxiling 
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domain.      In  other  words,   the   improved  communication   is    channeled  into 
the   schediiling  fimction   rather  than,   say,   the  technical  area.      This 
kind  of  organization   is   logical   from   a  motivational  viewpoint  becaxise 
the  lab  manager  is   probably  more   concerned  with  scheduling  than   is  the 
project  manager.      PERT  enables   the   lab  manager  to  communicate  his    con- 
cerns  to  the  project  manager. 

In  this   light,   the   establishment   of  priority   conflict   is   rather 
interesting.      (As   noted  before  the   IM   and  PM  each  think  that   the  other 
has   this   authority.)      If,   as  we  have   surmised,  the  LM  has  moved  into 
the  scheduling   area,   he   subsequently   ought    also  to  have   some   control 
over  priority.      However,   the   relationship  between  priority   and  completion 
is  not   direct,   and  it   is   possible  that  the   lab  manager  is  pushing  hard 
enoiJgh   in  this   area  so  that  the   PM   feels   that   the   LJ-t  has   the   authority 
although  the   U'[  is   not   quite   ready  to  admit   that  he   is   determining 
priorities.      VThile   "gaining"   authority  in  the  scheduling  area,   the   LM 
cedes   some   in   the   areas   of  subcontracting   and  overtime  budget.      This 
may  be   a  political  move  to  keep  the   PM  happy,  or  it  may  be   that  the   LM 
simply  does  not   have   time  to  do  both.      However,  the  net   effect   allows 
the  PM  two  more   tools  with  which   to   solve  the   technical   and  cost    "fires". 
Combining  this   information  with  Poust's   study,  we   find  that   this   shift 
brings  the  project  manager's   Job   closer  to  the   way  in  which  he   actually 
perceives   it. 

In  summary,  using  PERT  does   lead  to  better  cost/schedule  performance 
but  has   no  effect  on  technical  performance.      However,  we   suspect   that   the 
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reason   for  the  improvement   in   schedulinfr  performance   lies  not   in  the 
■fact   that   PERT  is   a  better  scheduling  technique,  but   that   it  provides 
a  method  by  which  the  person   in  the  organization  who  is   concerned  with 
scheduling    (in  this   case,   the  LM)    can  gain   authority   in  this   area. 
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